Filter device, filtering method and filtering program product

ABSTRACT

A filter device includes a vertical NR process portion, a vertical resolution conversion portion and a vertical NR process decision portion. The vertical NR process portion receives a horizontal NR processed signal and produces an NR processed signal. The vertical resolution conversion portion includes a plurality of resolution conversion filters and uses the NR processed signal to perform a vertical resolution conversion. The vertical NR process decision portion decides whether or not the vertical NR process portion should perform the vertical NR process on the horizontal NR processed signal in accordance with a resolution conversion filter selection signal for selecting one of plurality of resolution conversion filters for the vertical resolution conversion portion.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to a filter device, and more specificallyto a filter device using a noise reduction (NR) filter. The presentinvention also relates to a filtering method and a filtering programproduct using an NR filter.

2. Description of the Prior Art

Some electronic devices, including DVD recorders that handle a codedimage signal, use a noise reduction (NR) filter for reducing block noiseor mosquito noise so as to improve image quality. An NR filter that isused in a circuit after decoding a coded image signal is particularlycalled a postfilter.

(Filter Device 400)

FIG. 21 shows a block diagram of a filter device 400 for performing anoise reduction (NR) process.

The filter device 400 includes a postfilter portion 300 that performs ahorizontal and a vertical NR process on a decoded image signal 310 so asto produce an NR processed signal 312, and a vertical resolutionconversion portion 303 that performs a vertical resolution conversion ofthe NR processed signal 312 so as to produce a display image signal 313.

(Postfilter Portion 300)

The postfilter portion 300 includes a horizontal NR process portion 301that receives the decoded image signal 310 and produces a horizontal NRprocessed signal 311, and a vertical NR process portion 302 thatreceives the horizontal NR processed signal 311 and produces the NRprocessed signal 312.

The horizontal NR process portion 301 that performs the horizontal NRprocess on the decoded image signal 310 includes a condition decisionportion 304 and a horizontal NR process execution portion 305. Thecondition decision portion 304 decides the application conditions as towhether or not to apply the horizontal NR filter to the decoded imagesignal 310 in accordance with a previously set threshold level 315. Thehorizontal NR process execution portion 305 performs the horizontal NRprocess on the decoded image signal 310 in accordance with the decodedimage signal 310 and the result of the decision 318 by the conditiondecision portion 304 so as to produce the horizontal NR processed signal311.

The vertical NR process portion 302 that performs the vertical NRprocess on the horizontal NR processed signal 311 includes a conditiondecision portion 306 and a vertical NR process execution portion 307.The condition decision portion 306 decides whether or not to apply thevertical NR filter to the horizontal NR processed signal 311 inaccordance with a previously set threshold level 316. The vertical NRprocess execution portion 307 performs the vertical NR process on thehorizontal NR processed signal 311 in accordance with the horizontal NRprocessed signal 311 and the result of the decision 319 by the conditiondecision portion 306 so as to produce the NR processed signal 312.

FIG. 22 shows a block diagram for explaining the structure of a verticalNR filter portion that is used in the vertical NR process portion 302.The vertical NR process portion 302 includes six vertical NR filterportions 321-326, each of which has the number of filter taps [3]. Thevertical NR filter portions 321-326 are supplied with the horizontal NRprocessed signal 311. More specifically, pixel values Y(y−3) to Y(y+4)of eight lines at the same horizontal position of the horizontal NRprocessed signal 311 enter the vertical NR filter portions 321-326 ingroups of three lines.

Each of the vertical NR filter portions 321-326 decides whether or notthe vertical NR filter should be used for the horizontal NR processedsignal 311 in accordance with the input pixel values Y(y−3) to Y(y+4)and the set threshold level 316 so as to produce the NR processed signal312. The NR processed signal 312 is produced as pixel values Y′(y−2) toY′(y+3) by the vertical NR filter portions 321-326. Namely, each of thevertical NR filter portions 321-326 works as the condition decisionportion 306 and the vertical NR process execution portion 307.

The decision whether or not the filter is to be applied in each of thevertical NR filter portions 321-326 will be described with reference toFIG. 23. For example, the vertical NR filter portion 323 includes avertical NR filter having the number of filter taps [3] for using threepixels “y−1”, “y” and “y+1” for a filter application pixel “y” having apixel value Y(y). Whether the vertical NR filter should be used or notfor the filter application pixel “y” in the vertical NR filter portion323 is decided by using the filter application condition F1. If a valueof the filter application condition F1 is [1], a filter having filterfactor [1,2,1] is applied to pixels “y−1”, “y” and “y+1” so as toproduce a pixel value Y′ (y). Note that CMP(y) is a comparison functiondefined by the equation F3 in the filter application condition F1.Furthermore, TH in the equation F3 means the threshold level 316.

The basic operation is similar with respect to other vertical NR filterportions 321, 322 and 324-326. In addition, the basic operation is alsosimilar with respect to the horizontal NR filter portion in thehorizontal NR process portion 301.

(Vertical Resolution Conversion Portion 303)

The NR processed signal 312 produced by the postfilter portion 300 (seeFIG. 21) is received by the vertical resolution conversion portion 303.The received NR processed signal 312 is processed by the verticalresolution conversion filter for vertical resolution conversion and thedisplay image signal 313 is produced. The vertical resolution conversionportion 303 includes a plurality of vertical resolution conversionfilters that are applied in accordance with a conversion ratio for thevertical resolution. The plural vertical resolution conversion filtersare selected in accordance with a resolution conversion filter selectionsignal 317.

With reference to FIG. 24, three resolution conversion filters of thevertical resolution conversion portion 303 will be described. Thevertical resolution conversion portion 303 selects a resolutionconversion filter in accordance with three values [0], [1] and [2]designated by the resolution conversion filter selection signal 317. Ifthe value [0] is designated for example, the resolution conversion ofthe NR processed signal 312 is not performed, and a pixel value Y″(y)for Y″(y)=Y′(y) is generated as the display image signal 313. If thevalue [1] is designated, a pixel value Y″(y) that is derived from sixpixel values Y′(y−2) to Y′(y+3) of the NR processed signal 312 areproduced. Here, the value [1] is designated for example, when theconversion of the vertical resolution is not a conversion with a simpleconversion ratio such as an integer or an inverse number of an integer.If the value [2] is designated, a pixel value Y″(y) derived from fourpixel values Y′(y−1) to Y′(y+2) of the NR processed signal 312 isgenerated. Here, the value [2] is designated for example, when theconversion of the vertical resolution is a conversion with a simpleconversion ratio such as an integer or an inverse number of an integer.

(Block NR Filter)

On the other hand, there is a technique known in which one of aplurality filters are switched and used for the purpose of improving thenoise reduction effect, especially for the purpose of reducing blocknoise (see ISO/IEC, 14496-2: 2001(E), “Information Technology—Coding ofAudio-Visual Objects—Part 2: Visual”, Second Edition, Dec. 1, 2001, pp.448-450, for example).

With reference to FIG. 25, a deblock filter for reducing block noisethat is disclosed in the above-mentioned document will be described. Thedeblock filter disclosed in the above-mentioned document calculatesfiltered pixel values Y′(k+1) to Y′(k+8) for eight pixels “k+1” to “k+8”(whose pixel values are Y(k+1)−Y(k+8)) including block boundaries of 8×8pixels of an image block (for example, a coded block) that is a unit ofprocess of an image signal. More specifically, differences betweenneighboring pixels are calculated for the pixels “k+1” to “k+8”, so asto switch the filter by condition decision using the difference betweenneighboring pixels. Here, the condition decision is performed inaccordance with the equation F5. Note that φ(γ) is a comparison functiondefined by the equation F6 in the equation F5. Namely, it is decidedwhether an absolute value of the difference between neighboring pixelsis smaller than or equal to the threshold level THR1 in the equation F6.

If the value eq_cnt of the equation F5 is more than or equal tothreshold level THR2 and the equation F10 has a true value, the pixelvalues Y′(k+1) to Y′(k+8) after the filtering become values calculatedby the equation F11. Here, Ymax and Ymin in the equation F10 are themaximum value and the minimum value of the pixel values Y(k+1) toY(k+8), respectively. In addition, QP is a digitization parameter.Furthermore, p(k, i+j) and bj in the equation F11 are defined by theequations F12 and F13. Here, in the equation F13, “//” represents adivision operation in which the fractional portion is rounded. Note that“/” in the equation represents a division operation in which thefractional portion is dropped off(similar hereinafter).

On the other hand, if a value eq_cnt of the equation F5 is less than thethreshold level THR2, the pixel values Y′(k+1) to Y′(k+8) after thefiltering become values calculated by the equation F15. Here, “d” in theequation F15 becomes a value calculated by the equation F16. Inaddition, a3,0′ in the equation F16 has a value decided in accordancewith the equation F17 and the equation F18, while CLIP( ) is a functiondefined by the equation F19.

Concerning the above-described filter device, it is desirable to reducehardware costs and to reduce the process load of the filtering process.

In particular, it is desirable to reduce hardware costs when performingthe vertical NR process with hardware. When using a plurality of NRfilters that are switched in order to obtain a sufficient noisereduction effect, it is desirable to simplify the hardware structure orto simplify the processes by means of software.

SUMMARY OF THE INVENTION

An object of the present invention is to provide a filter device, afiltering method and a filtering program that can reduce hardware costsor software process loads.

A filter device according to a first aspect of the present inventionincludes an NR process unit, a resolution conversion unit and an NRprocess decision unit. The NR process unit receives an input imagesignal and generates an output image signal from the input image signal.The resolution conversion unit has a plurality of resolution conversionfilters and performs resolution conversion by using the output imagesignal. The NR process decision unit decides whether or not the NRprocess unit should perform an NR process on the input image signal inaccordance with a resolution conversion filter selection signal forselecting one of the plurality of resolution conversion filters for theresolution conversion unit.

The NR process is hereinafter defined as, for example, a smoothingprocess or the like in which a pixel in the input image signal that isto be subject to the NR process is smoothed with respect to other pixelsof the input image signal (the adjoining pixels in the vertical,horizontal, and other directions). In addition, the resolutionconversion filter selection signal is hereinafter defined as, forexample, a signal that is supplied in accordance with a resolutionconversion ratio necessary for an image size of the input image signaland a display size of the output image signal. The plurality ofresolution conversion filters are hereinafter defined to include, forexample, a filter that is not for resolution conversion and a resolutionconversion filter for producing one pixel for a display by using aplurality of pixels of the output image signal.

The filter device according to the present invention does not need aunit for performing the NR process on an output image signal, which isalways used in a state where the NR process is not performed amongoutput image signals used by the resolution conversion filter.Therefore, hardware costs can be reduced.

According to the filter device of a second aspect of the presentinvention, the NR process decision unit decides not to perform the NRprocess if the resolution conversion filter selection signal selects acomplicated resolution conversion filter.

The complicated resolution conversion filter means, for example, afilter that uses many pixels in the output image signal, or a filterwhose conversion ratio is not as simple as an integer or an inversenumber of an integer. The complicated resolution conversion filter isusually selected when there is little input image signal noise such as aperiod for playing DVD contents, for example. Therefore, good imagequality can be maintained even if the NR process is not performed on theinput image signal.

A filter device according to the present invention can maintain goodimage quality and enables reduction of hardware costs.

According to the filter device of a third aspect of the presentinvention, the NR process unit performs the NR process by using the samenumber of pixels of the input image signal as the maximum number ofpixels of the output image signal that are used by the plurality ofresolution conversion filters.

Conventionally, every output image signal is a signal after the NRprocess. Therefore, the number of pixels of the input image signal thatare used for obtaining the output image signal is larger than themaximum number of pixels of the output image signal that are used by theplurality of resolution conversion filters.

According to the filter device of the present invention, a portion ofthe output image signal is always the same as the input image signal.Therefore, it is possible to reduce the number of pixels of the inputimage signal that are used in order to obtain an output image signalhaving the same number of pixels as is conventional. Namely, it ispossible to reduce the number of memories for supplying input imagesignals necessary for the NR process and the number of signal lines forobtaining image signals from the memories.

According to the filter device of a fourth aspect of the presentinvention, the NR process unit has smaller number of NR filter unitsthan the maximum number of pixels of the output image signal that areused by the plurality of resolution conversion filters.

Here, the NR filter unit may be a filter that performs the NR process onthe input image signal by one NR filter or by switching among aplurality of NR filters, for example.

Conventionally, every output image signal is a NR processed signal.Therefore, the number of NR filters that are used for obtaining theoutput image signal is the same as the maximum number of pixels of theoutput image signal used by the plurality of resolution conversionfilters.

According to the filter device of the present invention, a portion ofthe output image signal is always the same as the input image signal.Therefore, it is possible to reduce the number of NR filters that areused in order to obtain the same number of output image signals as inconventional filtering methods or devices.

According to the filter device of a fifth aspect of the presentinvention, the NR process unit performs the NR process on the inputimage signal in the vertical direction, and the resolution conversionunit performs the resolution conversion of the output image signal inthe vertical direction.

The filter device of the present invention does not need a unit forperforming the NR process on an output image signal that is always usedin a state where the vertical NR process is not performed among outputimage signals that are used by the resolution conversion filter.Therefore, hardware costs can be reduced. For example, it is possible toreduce the number of line memories for supplying input image signalsnecessary for the NR process and the number of signal lines forobtaining image signals from the line memories.

A filtering method according to a sixth aspect of the present inventionis for producing an output image signal from an input image signal. Thefiltering method includes an NR process decision step and an NR processstep. The NR process decision step is for deciding whether or not toperform an NR process on the input image signal in accordance with aresolution conversion filter selection signal for selecting one of theplurality of resolution conversion filters for performing resolutionconversion of the output image signal. The NR process step is forperforming an NR process on the input image signal in accordance with adecision result in the NR process decision step.

The filter method according to the present invention does not need aunit for performing the NR process on an output image signal that isalways used in a state where the NR process is not performed amongoutput image signals that are used by the resolution conversion filter,so that hardware costs can be reduced. In addition, if the filteringmethod is performed by software, the software process load can bereduced.

A filtering program product according to a seventh aspect of the presentinvention is for use in a computer that performs a filtering method forproducing an output image signal from an input image signal. Thefiltering program product includes an NR process decision step fordeciding whether or not to perform an NR process on the input imagesignal in accordance with a resolution conversion filter selectionsignal for selecting one of the plurality of resolution conversionfilters for performing resolution conversion of the output image signal,and an NR process step for performing an NR process on the input imagesignal in accordance with a decision result by the NR process decisionstep.

The filtering program product according to the present invention doesnot need a step for performing the NR process on an output image signalthat is always used in a state where the NR process is not performed,amongst the output image signals that are used by the resolutionconversion filter, so the software process load can be reduced.

A filter device according to an eighth aspect of the present inventionincludes an NR process performing unit and an NR filter selection unit.The NR process performing unit has a plurality of NR filters. The NRfilter selection unit derives an image characterizing quantity by usingpixels in a range determined to correspond to a reference pixel rangethat is used by the plurality of NR filters, and selects one of theplurality of NR filters of the NR process performing unit in accordancewith each image characterizing quantity and a threshold level of theimage characterizing quantity that is set for each of the plurality ofNR filters.

The image characterizing quantity has a value that is determined inaccordance with a pixel value of a neighboring pixel in the vertical orthe horizontal direction. More specifically, it has a value determinedin accordance with a difference between neighboring pixels. Still morespecifically, the value is derived by calculating an absolute value ofthe difference between neighboring pixels or a square value of thedifference between neighboring pixels, for example.

According to the filter device of the present invention, it is possibleto use a common method of deriving the image characterizing quantity forselecting the NR filter. Thus, the process for selecting the NR filtercan be simplified. Namely, hardware costs for performing the process ofselecting the NR filter can be reduced.

According to the filter device of a ninth aspect of the presentinvention, each image characterizing quantity is derived by using pixelsin the reference pixel range.

According to the filter device of the present invention, it is possibleto use common pixels that are necessary for deriving the imagecharacterizing quantity and are used by each of the plurality of NRfilters. Therefore, hardware for storing or obtaining these pixels canbe commonly used.

According to the filter device of a tenth aspect of the presentinvention, a scale of the reference pixel range for each of theplurality of NR filters increases monotonously along with filterintensity, and the threshold level set for each of the plurality of NRfilters is decided so that the NR filter having larger filter intensitydoes not tend to be selected by the NR filter selection unit.

Here, the filter intensity means a degree of noise reduction, and pixelsbecome smoother by using a filter having a larger filter intensity.

According to the filter device of the present invention, it is possibleto use a filter having a larger filter intensity as the smoothness ofthe image increases. Therefore, an appropriate NR effect can beobtained.

According to the filter device of an eleventh aspect of the presentinvention, the NR filter selection unit gives higher priority to an NRfilter having a larger reference pixel range or an NR filter havinglarger filter intensity in selecting one of the plurality of NR filters.

According to the filter device of the present invention, it is possibleto use a filter having higher NR effect with a higher priority.

According to the filter device of a twelfth aspect of the presentinvention, the plurality of NR filters have the same sum of filterfactors.

According to the filter device of the present invention, it is possibleto use a common dividing process after adopting the filter factors ofthe plurality of NR filters. Therefore, hardware can be structured moresimply.

According to the filter device of a thirteenth aspect of the presentinvention, the NR filter selection unit selects the NR filter by using athreshold level that is set for a block boundary when a pixel that isused for deriving the image characterizing quantity is located on theblock boundary.

Here, the block boundary means a boundary between coded blocks when theinput image signal is coded, for example.

According to the filter device of the present invention, it is possibleto use a threshold level for the block boundary so that the NR filtercan be used more easily for pixels on the block boundary. Therefore,block noise can be reduced appropriately. In addition, if a pixel thatis used for deriving image characterizing quantity is located on theblock boundary, the NR filter can be used more easily even for a pixelthat is far from the block boundary. Therefore, block noise generated ina flat region (i.e, smooth region) can be reduced effectively.

According to the filter device of a fourteenth aspect of the presentinvention, the plurality of NR filters includes an NR filter for a blockboundary that is used only when a pixel that is a target of the NRprocess is located in the vicinity of the block boundary.

According to the filter device of the present invention, it is possibleto use an NR filter having a higher filter intensity for a pixel locatedon a block boundary. Therefore, block noise can be reduced moreappropriately.

According to the filter device of a fifteenth aspect of the presentinvention, the plurality of NR filters further includes an NR filter fora non-block boundary that is used only when a pixel that is a target ofthe NR process is not located in the vicinity of the block boundary.

According to the filter device of the present invention, it is possibleto use a more appropriate NR filter. More specifically, an NR filter fora non-block boundary such as a filter for reducing mosquito noise can beused if necessary.

A filtering method according to a sixteenth aspect of the presentinvention includes an NR filter selection step and an NR processexecution step. The NR filter selection step is for selecting one of theplurality of NR filters in accordance with an image characterizingquantity derived by using pixels in a range determined to correspond toa reference pixel range which each of the plurality of NR filters usesand a threshold level of the image characterizing quantity that is setfor each of the plurality of NR filters. The NR process execution stepis for performing an NR process on an image signal by using the selectedNR filter.

According to the filtering method of the present invention, it ispossible to use a common method for deriving the image characterizingquantity when selecting the NR filter. Thus, the process for selectingthe NR filter can be simplified. Namely, it is possible to reducehardware costs in order to perform a process for selecting the NR filteror to reduce the software process load for selecting the NR filter.

According to the filtering method of a seventeenth aspect of the presentinvention, the NR filter selection step gives higher priority to an NRfilter having a larger reference pixel range or an NR filter havinglarger filter intensity in selecting one of the plurality of NR filters.

According to the filtering method of the present invention, it ispossible to use a filter having a higher NR effect with a high priority.

According to the filtering method of an eighteenth aspect of the presentinvention, the NR filter selection step includes selecting the NR filterby using a threshold level that is set for a block boundary when a pixelthat is used for deriving the image characterizing quantity is locatedon the block boundary.

According to the filtering method of the present invention, it ispossible to use a threshold level for a block boundary so that the NRfilter can be used more easily for a pixel on a block boundary.Therefore, block noise can be reduced more appropriately.

A filtering program product according to a nineteenth aspect of thepresent invention is for use in a computer that performs an NR processon an image signal by using an NR filter selected from a plurality of NRfilters. The filtering program product includes an NR filter selectionstep for selecting one of the plurality of NR filters in accordance withan image characterizing quantity derived by using pixels in a rangedetermined to correspond to a reference pixel range which each of theplurality of NR filters uses and a threshold level of the imagecharacterizing quantity that is set for each of the plurality of NRfilters, and an NR process execution step for performing an NR processon an image signal by using the selected NR filter.

According to the filtering program product of the present invention, itis possible to use a common method for deriving the image characterizingquantity when selecting the NR filter. Thus, a process for selecting theNR filter can be simplified. Namely, the software process load forselecting the NR filter can be reduced.

According to the filtering program product of a twentieth aspect of thepresent invention, the NR filter selection step gives higher priority toan NR filter having a larger reference pixel range or an NR filterhaving a larger filter intensity in selecting one of the plurality of NRfilters.

According to the filtering program product of the present invention, itis possible to use a filter having a higher NR effect with a highpriority.

According to the filtering program product of a twenty-first aspect ofthe present invention, the NR filter selection step selects the NRfilter by using a threshold level that is set for a block boundary whena pixel that is used for deriving the image characterizing quantity islocated on the block boundary.

According to the filtering program product of the present invention, itis possible to use a threshold level for a block boundary so that the NRfilter can be used more easily for a pixel on a block boundary.Therefore, block noise can be reduced appropriately.

According to the present invention, a filter device, a filtering methodand a filtering program product can be provided that can reduce hardwarecosts or the software process load.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram of a filter device in a first embodiment ofthe present invention.

FIG. 2 is a block diagram of a vertical NR filter portion in the firstembodiment.

FIG. 3 is an explanatory diagram of a relationship between a resolutionconversion filter selection signal and a decision result of a verticalNR process decision portion in the first embodiment.

FIG. 4 is a flowchart of a filtering method according to the firstembodiment.

FIG. 5 is a block diagram of a filter device in a second embodiment ofthe present invention.

FIG. 6 is an explanatory diagram of a filter application condition inthe second embodiment.

FIG. 7 is an explanatory diagram of a horizontal NR filter in the secondembodiment.

FIG. 8 shows a hardware structure of a condition decision portion in thesecond embodiment.

FIG. 9 shows a hardware structure of a horizontal NR process executionportion in the second embodiment.

FIG. 10 is an explanatory diagram of a vertical NR filter in the secondembodiment.

FIG. 11 shows a hardware structure of a condition decision portion inthe second embodiment.

FIG. 12 shows a hardware structure of a vertical NR process executionportion in the second embodiment.

FIG. 13 is a flowchart of a filtering method in the second embodiment.

FIG. 14 is an explanatory diagram of a horizontal NR filter in a thirdembodiment of the present invention.

FIG. 15 shows a hardware structure of a condition decision portion inthe third embodiment.

FIG. 16 is a flowchart of a filtering method in the third embodiment.

FIG. 17 is a block diagram of the overall structure of a contentproviding system in a fourth embodiment of the present invention.

FIG. 18 shows an example of a cellular phone equipped with aninterpolation frame generation device according to the present inventionin the fourth embodiment.

FIG. 19 is a block diagram of the cellular phone in the fourthembodiment.

FIG. 20 shows an example of a digital broadcasting system in the fourthembodiment.

FIG. 21 is a block diagram of a filter device in the background art.

FIG. 22 is a block diagram of a vertical NR filter portion in thebackground art.

FIG. 23 is an explanatory diagram of a filter application condition inthe background art.

FIG. 24 is an explanatory diagram of a resolution conversion filter inthe background art.

FIG. 25 is an explanatory diagram of a conventional deblock filter inthe background art.

DESCRIPTION OF THE INVENTION

Hereinafter, the present invention will be explained in more detail withreference to embodiments and drawings.

FIRST EMBODIMENT

A filter device 10 according to a first embodiment of the presentinvention will be described with reference to FIGS. 1-4. The filterdevice 10 shown in FIG. 1 is characterized in that it will decidewhether or not a vertical NR process should be performed in accordancewith a selection signal of a vertical resolution conversion filter.

(Structure of the Filter Device 10)

The filter device 10 includes a postfilter portion 11 for performing ahorizontal and vertical NR process on a decoded image signal 25 so as toproduce an NR processed signal 27, a vertical resolution conversionportion 12 for performing a vertical resolution conversion on the NRprocessed signal 27 so as to produce a display image signal 28, and avertical NR process decision portion 20 for deciding whether or not thepostfilter portion 11 should perform a vertical NR process.

The postfilter portion 11 includes a horizontal NR process portion 13that receives the decoded image signal 25 and produces a horizontal NRprocessed signal 26, and a vertical NR process portion 14 that receivesthe horizontal NR processed signal 26 and produces the NR processedsignal 27.

The horizontal NR process portion 13 is a portion for performing ahorizontal NR process on the decoded image signal 25, which includes acondition decision portion 15 and a horizontal NR process executionportion 16. The condition decision portion 15 receives the decoded imagesignal 25 as a first input and a set threshold level 30 as a secondinput so as to produce a decision result 37 as to whether or not ahorizontal NR filter is used for the decoded image signal 25. Thehorizontal NR process execution portion 16 receives the decoded imagesignal 25 as a first input and the decision result 37 as a second inputso as to produce the horizontal NR processed signal 26 that is obtainedby performing the horizontal NR process corresponding to the decisionresult 37 on the decoded image signal 25.

The vertical NR process portion 14 is a portion for performing avertical NR process on the horizontal NR processed signal 26, whichincludes a condition decision portion 17 and a vertical NR processexecution portion 18. The condition decision portion 17 receives thehorizontal NR processed signal 26 as a first input, receives a setthreshold level 31 as a second input and receives the decision result 33of the vertical NR process decision portion 20 as a third input so as toproduce a decision result 38 about whether or not a vertical NR filteris used for the horizontal NR processed signal 26. The vertical NRprocess execution portion 18 receives the horizontal NR processed signal26 as a first input and the decision result 38 as a second input so asto produce the NR processed signal 27 that is obtained by performing thevertical NR process corresponding to the decision result 38 on thehorizontal NR processed signal 26.

The vertical resolution conversion portion 12 receives the NR processedsignal 27 as a first input and the resolution conversion filterselection signal 32 as a second input so as to produce a display imagesignal 28 that is obtained by performing the resolution conversionprocess corresponding to the resolution conversion filter selectionsignal 32 on the NR processed signal 27.

The vertical NR process decision portion 20 receives the resolutionconversion filter selection signal 32 and produces the decision result33 about whether or not the postfilter portion 11 performs the verticalNR process.

(Operation of the Filter Device 10)

The horizontal NR process portion 13 performs the horizontal NR processon the decoded image signal 25. The condition decision portion 15decides an application condition as to whether or not the horizontal NRfilter should be used for the decoded image signal 25 in accordance withthe set threshold level 30. The horizontal NR process execution portion16 performs the horizontal NR process on the decoded image signal 25 inaccordance with the decoded image signal 25 and the decision result ofthe condition decision portion 15, so as to produce the horizontal NRprocessed signal 26.

The vertical NR process portion 14 performs the vertical NR process onthe horizontal NR processed signal 26. The condition decision portion 17decides an application condition as to whether or not the vertical NRfilter should be used for the horizontal NR processed signal 26 inaccordance with the set threshold level 31 and the decision result 33 ofthe vertical NR process decision portion 20. The vertical NR processexecution portion 18 performs the vertical NR process on the horizontalNR processed signal 26 in accordance with the horizontal NR processedsignal 26 and the decision result of the condition decision portion 17,so as to produce the NR processed signal 27.

FIG. 2 shows a block diagram of a vertical NR filter portion that isused for the vertical NR process portion 14. The vertical NR processportion 14 receives the horizontal NR processed signal 26. Thehorizontal NR processed signal 26 is supplied as pixel values Y(y−2) toY(y+3) of six lines at the same position in the horizontal directionfrom line memory (not shown) which stores the horizontal NR processedsignal 26 of the corresponding line. Note that the line memory (notshown) may store the decoded image signal 25 instead of the horizontalNR processed signal 26, and the horizontal NR processed signal 26 may besupplied directly to the vertical NR process portion and not via theline memory (not shown). Furthermore, instead of the line memory (notshown), a frame memory (not shown) may be used that can simultaneouslyobtain a plurality of lines of pixels at the same position in thehorizontal direction.

The vertical NR process portion 14 includes four vertical NR filterportions 41-44. Each of the vertical NR filter portions 41-44 has avertical NR filter whose number of filter taps is [3], which is suppliedwith every three lines of pixel values Y(y−2) to Y(y+3).

The vertical NR filter portions 41-44 perform a filtering process inaccordance with the supplied pixel values Y(y−2) to Y(y+3), the setthreshold level 31 and the decision result 33 of the vertical NR processdecision portion 20, so as to produce the NR processed signal 27 havingpixel values Y′(y−1) to Y′(y+2) after the filtering process.

More specifically, if the decision result 33 of the vertical NR processdecision portion 20 decides that the vertical NR filters should not beused (i.e., filter through), the pixel value Y′(y) that is identical toY(y) will be the output of the vertical NR filter portions 41-44. On theother hand, if the decision result 33 is that the vertical NR filtershould be used, each of the vertical NR filter portions 41-44 performsthe filtering process in the same way as described above with referenceto FIG. 23. For example, the vertical NR filter portion 42 has avertical NR filter whose number of filter taps is [3] that uses threepixels of “y−1”, “y” and “y+1” for the filter application pixel “y” ofthe pixel value Y(y). Whether the vertical NR filter should be used forthe filter application pixel “y” is decided by using the filterapplication condition F1 shown in FIG. 23.

The filter application condition F1 shown in FIG. 23 is a function of aconstant parameter TH_d and a comparison function CMP(y). The filterapplication condition F1 is a function that produces a value indicatingthat the filter should be used (for example, a value [1]) when thefilter application pixel “y” is decided to be included in a flat region(smooth region) of an image in accordance with an image characterizingquantity F2 or a constant parameter. More specifically, it produces avalue indicating that the filter should be used by comparing a result ofone of four fundamental operations of arithmetic or other operationbetween the comparison functions CMP(y−1) and CMP(y) with the thresholdlevel TH_d. Note that the filter application condition F1 may indicatethat the filter should be used when the result of one of fourfundamental operations of arithmetic or other operation becomes aspecific value.

Here, the image characterizing quantity is, for example, a valueindicating a flatness (smoothness) of the filter application pixel “y”and is determined in accordance with a difference between neighboringpixels. More specifically, it is determined by calculating an absolutevalue of the difference between neighboring pixels or a square value ofthe difference between neighboring pixels, for example. Furthermore, thecomparison function CMP(y) compares the threshold level with the imagecharacterizing quantity, so as to produce a value indicating thecomparison result. For example, CMP(y) produces a value [1] when thethreshold level is larger.

If a value of the filter application condition F1 is [1], a filter of afilter factor [1,2,1] is used for the pixels of “y−1”, “y” and “y+1”, sothat the pixel value Y′(y) is produced. Namely, each of the vertical NRfilter portions 41-44 works as the condition decision portion 17 as wellas the vertical NR process execution portion 18.

On the other hand, in the horizontal NR processed signal 26 received bythe vertical NR process portion 14, the pixel value Y(y−2) and the pixelvalue Y(y+3) are delivered as the NR processed signal 27 without anychange.

The vertical resolution conversion portion 12 performs a verticalresolution conversion of the NR processed signal 27 that is suppliedfrom the postfilter portion 11 and produces the display image signal 28.The vertical resolution conversion portion 12 includes a plurality ofvertical resolution conversion filters that are used corresponding to aconversion ratio of the vertical resolution. One of the plurality ofvertical resolution conversion filters is selected in accordance withthe resolution conversion filter selection signal 32. Operations of theresolution conversion filter selection signal 32 and each of thevertical resolution conversion filters are the same as described abovewith reference to FIG. 24. Namely, the vertical resolution conversionportion 12 selects the resolution conversion filter in accordance withthree values [0]-[2] designated by the resolution conversion filterselection signal 32. For example, if the value [0] is designated, theresolution conversion is not performed while the pixel value Y″(y) thatis identical to Y′(y) is produced as the display image signal 28. If thevalue [1] is designated, the pixel value Y″(y) that is derived from sixpixel values Y(y−2), Y′(y−1) to Y′(y+2) and Y(y+3) is delivered. Here,the value [1] is designated when the vertical conversion of resolutionis not a conversion having a simple conversion ratio such as an integeror an inverse number of an integer but is a conversion having largenumber of filter taps or when using a filter having large number of tapsfor high image quality in a conversion having a simple conversion ratio,for example. If the value [2] is designated, the pixel value Y″(y) thatis derived from four pixel values Y′(y−1) to Y′(y+2) is delivered. Here,the value [2] is designated when the vertical conversion of resolutionhas a simple conversion ratio such as an integer or an inverse number ofan integer and the number of filter taps is small, for example.

Here, the resolution conversion filter selection signal 32 is obtainedfrom a decoding device that performs decoding of the decoded imagesignal 25 (see FIG. 1). More specifically, when the decoding deviceperforms the decoding of the decoded image signal 25, the conversionratio of resolution is determined from the image size, the aspect ratioor the like of the decoded image signal 25, so that the resolutionconversion filter selection signal 32 is supplied in accordance withthis.

The vertical NR process decision portion 20 decides whether or not thevertical NR filter should be used in accordance with the resolutionconversion filter selection signal 32. FIG. 3 shows the relationshipbetween the resolution conversion filter selection signal 32 and thedecision result 33 of the vertical NR process decision portion 20. Ifthe resolution conversion filter selection signal 32 indicates the value[1], the vertical NR process decision portion 20 decides not to use thevertical NR filter (filter through) and delivers the value [1] as thedecision result 33 (see FIG. 3( a)). If the resolution conversion filterselection signal 32 indicates the value [0] or the value [2], thevertical NR process decision portion 20 decides to use the vertical NRfilter and delivers the value [0] as the decision result 33 (see FIG. 3(b)).

Note that the values of the resolution conversion filter selectionsignal, the decision result, the filter application condition F1 andothers are not limited to the values shown above.

(Filtering Method)

FIG. 4 shows a flowchart of a filtering method in the filter device 10,particularly a filtering method in the vertical NR process decisionportion 20 and the vertical NR process portion 14.

This filtering method is a process that is performed for each pixel ofthe NR processed signal 27 that is delivered from the vertical NR filterportions 41-44 (steps S11-S14). The vertical NR process decision portion20 obtains the resolution conversion filter selection signal 32 (stepS11) and decides whether or not the vertical NR process portion 14should perform the vertical NR process (step S12). The vertical NRprocess portion 14 performs the vertical NR process of the horizontal NRprocessed signal 26 (step S14) when the decision result 33 indicates thevalue [0] (step S13). On the other hand, if the decision result 33indicates the value [1] (step S13), the vertical NR process portion 14does not perform the vertical NR process on the horizontal NR processedsignal 26 and starts the process on the next filter application pixel(steps S11-S14).

The filtering method described above can be realized by a program usinga computer without being limited to the filter device 10.

(Effects of the First Embodiment)

(1) It is not necessary to provide a vertical NR filter portion forY(y−2) and Y(y+3) that are always used in a state in which the verticalNR process is not performed among the NR processed signal 27 that isused by the vertical resolution conversion portion 12. Namely, hardwarecosts can be reduced compared with the conventional vertical NR processportion 302 (see FIG. 22).

(2) If the vertical conversion of resolution is not a conversion havinga simple conversion ratio such as an integer or an inverse number of aninteger, a complicated resolution conversion filter having a largenumber of taps is selected. Here, when a complicated conversion ratio isrequired, such as a conversion from the aspect ratio of 4:3 to theaspect ratio 16:9, there is usually little noise in the decoded imagesignal 25, like when playing DVD contents. Therefore, even if thevertical NR process is not performed on the horizontal NR processedsignal 26, the image quality of the display image signal 28 can bemaintained.

(3) In the conventional vertical NR process portion 302, the NRprocessed signal 312 can be always processed by the vertical NR process.Therefore, for example, the horizontal NR processed signal 311 of eightlines is necessary for delivering the NR processed signal 312 of sixlines (see FIG. 22). Namely, the number of lines of the horizontal NRprocessed signal 311 that is necessary for obtaining the NR processedsignal 312 is larger than the number of lines of the NR processed signal312.

On the other hand, in the vertical NR process portion 14 according tothe present invention, a portion of the NR processed signal 27 (pixelvalues Y(y−2) and Y(y+3) in FIG. 2) is always identical to a portion ofthe horizontal NR processed signal 26. Therefore, in order to obtain theNR processed signal 27 of six lines that is similar to that of aconventional device, the horizontal NR processed signal 26 of six linesis necessary. Namely, the number of lines of the horizontal NR processedsignal 26 that is necessary for obtaining the NR processed signal 27 isthe same as the number of lines of the NR processed signal 27, so thatthe number of lines of the horizontal NR processed signal 26 can bereduced in order to obtain the NR processed signal 27 of the same numberof lines as a conventional device. As a result, the number of linememories for obtaining the horizontal NR processed signal 26 needed forthe vertical NR process can be reduced.

(4) In the conventional vertical NR process portion 302, the NRprocessed signal 312 can be always processed by the vertical NR process.Therefore, the number of the vertical NR filter portions that are usedfor obtaining the NR processed signal is equal to the number of lines ofthe NR processed signal 312, i.e., the maximum number of lines of the NRprocessed signal 312 that is used for the vertical resolution conversionby the vertical resolution conversion portion 303.

On the other hand, in the vertical NR process portion 14 according tothe present invention, a portion of the NR processed signal 27 is alwaysidentical to a portion of the horizontal NR processed signal 26.Therefore, the number of the vertical NR filter portions that arenecessary to obtain the NR processed signal 27 of six lines similar tothat of a conventional device is four, so that the number of thevertical NR filter portions can be reduced.

(Modifications)

(1) The vertical NR process portion 14 may perform the NR process aplurality of times using one vertical NR filter portion instead of theplurality of vertical NR filter portions 41-44.

In this case, the quantity of processing for the NR process can bereduced.

(2) The condition decision portion 17 may decide the condition by usingthe decoded image signal 25. In this case, the condition decisionportion 17 becomes able to work in parallel with the horizontal NRprocess before the horizontal NR process is completed, so that theentire process speed of the filter device 10 can be improved.

(3) It is possible to adopt a structure for the horizontal NR processportion 13 that is the same as described in the above embodiment.

SECOND EMBODIMENT

The horizontal and vertical NR filter of the filter device can be onethat is selected from a plurality of candidate filters. This NR filterwill be described with reference to FIGS. 5-13.

(Structure of a Filter Device 50)

FIG. 5 shows a filter device 50 having a horizontal and vertical NRfilter that is one selected from a plurality of candidate filters.

The filter device 50 includes a postfilter portion 51 for performing ahorizontal and vertical NR process on the decoded image signal 65 so asto deliver a NR processed signal 67, and a vertical resolutionconversion portion 52 for performing the vertical resolution conversionon the NR processed signal 67 so as to deliver a display image signal68.

The postfilter portion 51 includes a horizontal NR process portion 53that receives the decoded image signal 65 and produces a horizontal NRprocessed signal 66, and a vertical NR process portion 54 that receivesthe horizontal NR processed signal 66 and produces the NR processedsignal 67.

The horizontal NR process portion 53, which is a portion for performingthe horizontal NR process on the decoded image signal 65, includes acondition decision portion 55 and a horizontal NR process executionportion 56. The condition decision portion 55 receives the decoded imagesignal 65 as a first input and a set threshold level 70 as a secondinput, selects the horizontal NR filter that is used for the decodedimage signal 65, and produces a selection result 74. The horizontal NRprocess execution portion 56 receives the decoded image signal 65 as afirst input and the selection result 74 as a second input so as toproduce the horizontal NR processed signal 66 that is obtained by thehorizontal NR process on the decoded image signal 65 corresponding tothe selection result 74.

The vertical NR process portion 54, which is a portion for performingthe vertical NR process on the horizontal NR processed signal 66,includes a condition decision portion 57 and a vertical NR processexecution portion 58. The condition decision portion 57 receives thehorizontal NR processed signal 66 as a first input and a set thresholdlevel 71 as a second input, selects the vertical NR filter that is usedfor the horizontal NR processed signal 66, and produces a selectionresult 75. The vertical NR process execution portion 58 receives thehorizontal NR processed signal 66 as a first input and the selectionresult 75 as a second input so as to deliver the NR processed signal 67that is obtained by the vertical NR process on the horizontal NRprocessed signal 66 corresponding to the selection result 75.

The vertical resolution conversion portion 52 receives the NR processedsignal 67 as a first input and the resolution conversion filterselection signal 72 as a second input so as to deliver the display imagesignal 68 that is obtained by the resolution conversion process on theNR processed signal 67 corresponding to the resolution conversion filterselection signal 72.

(NR Filter and NR Filter Selection)

The NR filters of the horizontal NR process portion 53 and the verticalNR process portion 54, and selection of the NR filter will now bedescribed. According to the present invention, each of the horizontal NRprocess portion 53 and the vertical NR process portion 54 has aplurality of NR candidate filters and performs the NR process on afilter application pixel by selecting one NR filter from the pluralityof NR candidate filters.

The plurality of candidate filters FH1-FH3, . . . have the numbers offilter taps FH1tap-FH3tap, . . . and threshold levels TH1-TH3, . . .used by the filters, respectively. Here, it is assumed that filterintensity of the candidate filter FHk is smaller as the value of kincreases. In addition, it is assumed that the number of filter tapsFHktap decreases monotonously along with k, and each of the thresholdlevels THk and TH_dk increases monotonously along with k. Here, it isassumed that the candidate filter having larger threshold levels THk andTH_dk tends to be selected. Namely, as the value of k increases, thecandidate filter FHk tends to be selected.

FIG. 6 shows a filter application condition F30 for deciding whether ornot the filter FHk should be used for a filter application pixel “x”having a pixel value Y(x). The filter application condition F30 is afunction of a constant parameter TH_dk and a comparison function CMPk(x)(see the equation F32). The filter application condition F30 is afunction that produces a value (for example, the value [1]) thatindicates that the filter should be used when the filter applicationpixel “x” is decided to be included in a flat region (smooth region) ofthe image in accordance with an image characterizing quantity DIFF(x) ora constant parameter THk. More specifically, a result of one of fourfundamental operations of arithmetic or other operation of values of thecomparison function CMPk(x) is compared with the threshold level TH_dkso that the value is produced that indicates that the filter should beused. Note that the filter application condition F30 may be that thefilter is used when a result of one of four fundamental operations ofarithmetic or other operation of values of the comparison functionCMPk(x) becomes a specific value.

Here, the image characterizing quantity DIFF(x) is a value that isdefined by the equation F31 and indicates a flatness (smoothness) of thefilter application pixel “x”, for example. More specifically, it isdetermined by calculating an absolute value of the difference betweenneighboring pixels or a square value of the difference betweenneighboring pixels, for example. Furthermore, THk and TH_dk arethreshold levels for each of the candidate filters. Furthermore, thecomparison function CMPk(x) compares the threshold level with the imagecharacterizing quantity, so as to produce a value indicating thecomparison result. For example, CMPk(y) produces a value [1] when thethreshold level is larger.

If a value of the filter application condition F30 is [1], the filterFHk is used for the filter application pixel “x”. Furthermore, if thereare a plurality of filters FHk in which a value of the filterapplication condition F30 becomes [1], one having a smaller value of k,i.e., one having larger filter intensity or one having a larger numberof filter taps is selected with higher priority.

(Structure of the Horizontal NR Process Portion 53)

The NR filter shown in FIG. 6 and a specific hardware structure forrealizing selection of the NR filter will be described with reference toFIGS. 7-9. The horizontal NR process portion 53 selects the NR filterand performs the NR process on the decoded image signal 65 that issequentially input by pixel position.

FIG. 7 shows numbers of filter taps, filter factors and threshold levelsof three horizontal NR filters FH1-FH3 that are used and switched in thehorizontal NR process portion 53. The horizontal NR filter FH1 has thenumber of filter taps FH1tap=[7], the filter factor [1, 1, 1, 2, 1, 1,1], and the threshold level TH1=[8]. The horizontal NR filter FH2 hasthe number of filter taps FH2tap=[5], the filter factor [1, 1, 4, 1, 1],and the threshold level TH2=[16]. The horizontal NR filter FH3 has thenumber of filter taps FH3tap=[3], the filter factor [2,4,2], and thethreshold level TH3=[32].

FIG. 8 shows a hardware structure of the condition decision portion 55.The condition decision portion 55 receives the decoded image signal 65as a first input and the set threshold level 70 (TH1-TH3) as a secondinput, selects horizontal NR filters FH1-FH3 that are used for thedecoded image signal 65, and delivers the selection result 74. Thecondition decision portion 55 includes delay elements 80 a-80 f of onepixel delay, computing elements 81 a-81 f for calculating the imagecharacterizing quantity in accordance with pixel values of neighboringpixels, comparators 82 a-82 l for realizing the comparison functionCMPk(x) defined by the equation F32, decision elements 83 a-83 c forrealizing the filter application condition F30, and AND gates 83 d and83 e for selecting one horizontal NR filter that is used for thefiltering process from horizontal NR filters FHk that satisfy the filterapplication condition F30.

The delay elements 80 a-80 f are connected to the decoded image signal65 in series for one pixel delay, and produce pixel values Y(x+2) toY(x−3) of the past six pixels from the pixel value Y(x+3) received asthe decoded image signal 65.

Each of the computing elements 81 a-81 f calculates the imagecharacterizing quantity from the input and the output of each of thedelay elements 80 a-80 f. Namely, the computing elements 81 a-81 fcalculate the image characterizing quantities DIFF(x+2) to DIFF(x−3)defined by the equation F31 for the pixel values Y(x+3) to Y(x−3),respectively.

Each of comparators 82 a-82 f receives one of the image characterizingquantities DIFF(x+2) to DIFF(x−3) as a first input and the thresholdlevel TH1 of the horizontal NR filter FH1 as a second input so as toproduce one of the comparison results. Namely, each of the comparators82 a-82 f compares one of the image characterizing quantities DIFF(x+2)to DIFF(x−3) with the threshold level TH1 of the horizontal NR filterFH1 so as to deliver the comparison result to the decision element 83 a.Each of comparators 82 g-82 j receives one of the image characterizingquantities DIFF(x+1) to DIFF(x−2) as a first input and the thresholdlevel TH2 of the horizontal NR filter FH2 as a second input so as toproduce one of the comparison results. Namely, each of the comparators82 g-82 j compares one of the image characterizing quantities DIFF(x+1)to DIFF(x−2) with the threshold level TH2 of the horizontal NR filterFH2 so as to deliver the comparison result to the decision element 83 b.Each of comparators 82 k and 82 l receives one of the imagecharacterizing quantities DIFF(x) and DIFF(x−1) as a first input and thethreshold level TH3 of the horizontal NR filter FH3 as a second input soas to produce one of the comparison results. Namely, each of thecomparators 82 k and 82 l compares one of the image characterizingquantities DIFF(x) and DIFF(x−1) with the threshold level TH3 of thehorizontal NR filter FH3 so as to deliver the comparison result to thedecision element 83 c.

The decision element 83 a receives outputs of the comparators 82 a-82 fand the threshold level TH_d1 for these outputs so as to calculate thefilter application condition F30. More specifically, if the pixels “x+3”to “x−3” are decided to be in a flat region (smooth region) for example,the value [1] is delivered so as to indicate that the horizontal NRfilter FH1 satisfies the filter application condition F30. The decisionelement 83 b receives outputs of the comparators 82 g-82 j and thethreshold level TH_d2 for these outputs so as to calculate the filterapplication condition F30. More specifically, if the pixels “x+2” to“x−2” are decided to be in a flat region (smooth region), the value [1]is delivered so as to indicate that the horizontal NR filter FH2satisfies the filter application condition F30. The decision element 83c receives outputs of the comparators 82 k and 82 l and the thresholdlevel TH_d3 for these outputs so as to calculate the filter applicationcondition F30. More specifically, if the pixels “x+1” to “x−1” aredecided to be in a flat region (smooth region) for example, the value[1] is delivered so as to indicate that the horizontal NR filter FH3satisfies the filter application condition F30.

Here, the output of the decision element 83 a is delivered as theselection result 74 to the horizontal NR process execution portion 56.Namely, if the output of the decision element 83 a indicates the value[1], it means that the horizontal NR filter FH1 is used for the filterapplication pixel “x”. The output of the decision element 83 b isreceived by the AND gate 83 d. The AND gate 83 d receives negation ofthe output value of the decision element 83 a as a first input and theoutput value of the decision element 83 b as a second input so as todeliver the selection result 74. Namely, if the output of the decisionelement 83 a indicates the value [0] and the output of the decisionelement 83 b indicates the value [1], the AND gate 83 d delivers thevalue [1] so as to indicate that the horizontal NR filter FH2 is usedfor the filter application pixel “x”. The output of the decision element83 c is received by the AND gate 83 e. The AND gate 83 e receivesnegation of the output value of the decision element 83 a as a firstinput, negation of the output value of the decision element 83 b as asecond input and the output value of the decision element 83 c as athird input so as to deliver the selection result 74. Namely, if theoutputs of the decision elements 83 a and 83 b indicate the value [0]and the output of the decision element 83 c indicates the value [1], theAND gate 83 e delivers the value [1] so as to indicate that thehorizontal NR filter FH3 is used for the filter application pixel “x”.As described above, the selection result 74 is an output of a value thatindicates whether each of the horizontal NR filters FH1-FH3 is used ornot.

FIG. 9 shows a hardware structure of the horizontal NR process executionportion 56. The horizontal NR process execution portion 56 receives thedecoded image signal 65 as a first input and the selection result 74 asa second input, so as to produce the horizontal NR processed signal 66that is obtained by performing the horizontal NR process on the decodedimage signal 65 corresponding to the selection result 74. The horizontalNR process execution portion 56 includes delay elements 85 a-85 f forone pixel delay, adders 87 a-87 f for performing adding operation ofsignals, multipliers 88 a-88 d for performing multiplying operation ofthe filter factors, switches 86 a-86 d for designating the filterfactors, and a divider 90 for performing dividing operation of a valueof sum of products of the filter factor and the decoded image signal 65.

The delay elements 85 a-85 f are connected to the decoded image signal65 in series for one pixel delay, and produce pixel values Y(x+2) toY(x−3) of the past six pixels from the pixel value Y(x+3) received asthe decoded image signal 65. The adders 87 a-87 c perform addingoperation of pixel values having the same filter factor that aremultiplied even if any one of the horizontal NR filters FH1-FH3 is used.The adder 87 a adds the pixel value Y(x+3) and the pixel value Y(x−3).The adder 87 b adds the pixel value Y(x+2) and the pixel value Y(x−2).The adder 87 c adds pixel value Y(x+1) and the pixel value Y(x−1).

The multipliers 88 a-88 d multiply the outputs of adders 87 a-87 c andthe delay element 85 c by the filter factor of the horizontal NR filterindicated by the selection result 74 of the condition decision portion55. The multiplier 88 a receives the output of the adder 87 a as a firstinput and the output of the switch 86 a as a second input so as tomultiply these inputs. The multiplier 88 b receives the output of theadder 87 b as a first input and the output of the switch 86 b as asecond input so as to multiply these inputs. The multiplier 88 creceives the output of the adder 87 c as a first input and the output ofthe switch 86 c as a second input so as to multiply these inputs. Themultiplier 88 d receives the output of the delay element 85 c (pixelvalue Y(x)) as a first input and the output of the switch 86 d as asecond input so as to multiply these inputs.

The switches 86 a-86 d operate responding to the selection result 74 ofthe condition decision portion 55, and the operation of the switchdesignates the filter factor that is multiplied to the output of theadders 87 a-87 c or to the delay element 85 c. The switches 86 a-86 ddesignate the upper filter factor if the horizontal NR filter FH1 isused, designate the middle filter factor if the horizontal NR filter FH2is used, and designate the lower filter factor if the horizontal NRfilter FH3 is used.

The adders 87 d-87 f add all outputs of the multipliers 88 a-88 d. Theadding result is divided by the sum of filter factors in the divider 90.Here, the sum of filter factors is set to be the value [8] if any one ofthe horizontal NR filters FH1-FH3 is selected. Therefore, the divider 90can perform the dividing operation by shifting operation of three bits.The dividing result of the divider 90 is delivered as the horizontal NRprocessed signal 66.

(Structure of the Vertical NR Process Portion 54)

The NR filter shown in FIG. 6 and a specific hardware structure forrealizing selection of the NR filter will be described with reference toFIGS. 10-12. The vertical NR process portion 54 stores the horizontal NRprocessed signal 66 that is output by line position from the horizontalNR process portion 53 in a line memory (not shown) for each line, andselection of the NR filter and the NR process are performed for thefilter application pixel included in the stored line. Note that the linememory (not shown) may store the decoded image signal 25 instead of thehorizontal NR processed signal 26, and the horizontal NR processedsignal 26 may be supplied to the vertical NR process portion not via theline memory (not shown). Furthermore, instead of the line memory (notshown), a frame memory (not shown) may be used that can obtainsimultaneously a plurality of lines of pixels at the same position inthe horizontal direction.

The vertical NR process portion 54 includes a plurality of vertical NRfilter portions. The number of vertical NR filter portions included inthe vertical NR process portion 54 depends on the number of lines afterthe vertical NR process that the vertical resolution conversion portion52 uses for the vertical resolution conversion process. For example, ifthe vertical resolution conversion portion 52 needs six lines at mostafter the vertical NR process similar to the vertical resolutionconversion portion 303 shown in FIG. 22, the number of the vertical NRfilter portion is six.

Hereinafter, the vertical NR filter portion for the filter applicationpixel “y” of the pixel value Y(y) will be described. Similar explanationfor other vertical NR filter portions is omitted.

FIG. 10 shows numbers of filter taps, filter factors and thresholdlevels of three vertical NR filters FH1-FH3 that are used and switchedin the vertical NR process portion 54. The vertical NR filter FH1 hasthe number of filter taps FH1tap=[3], the filter factor [3,2,3], and thethreshold level TH1=[8]. The vertical NR filter FH2 has the number offilter taps FH2tap=[3], the filter factor [2,4,2], and the thresholdlevel TH2=[16]. The vertical NR filter FH3 has the number of filter tapsFH3tap=[3], the filter factor [1,6,1], and the threshold level TH3=[32].

FIG. 11 shows a hardware structure of the condition decision portion 57.The condition decision portion 57 receives the horizontal NR processedsignal 66 that is stored by line position as a first input and the setthreshold level 71 (TH1-TH3) as a second input, selects the vertical NRfilters FH1-FH3 that are used for the horizontal NR processed signal 66,and delivers the selection result 75. The horizontal NR processed signal66 stored by line position and which is the first input includes threepixel values Y(y−1) to Y(y+1) stored in three line memories (not shown)at the same position in the horizontal direction.

The condition decision portion 57 includes computing elements 93 a and93 b for calculating the image characterizing quantity defined by theequation F31, comparators 94 a-94 f for realizing the comparisonfunction CMPk(x) defined by the equation F32, decision elements 95 a-95c for realizing the filter application condition F30, and AND gates 95 dand 95 e for selecting one vertical NR filter that is used for thefiltering process from vertical NR filters FHk that satisfy the filterapplication condition F30.

Each of the computing elements 93 a and 93 b calculates the imagecharacterizing quantity from the pixels “y−1” and “y+1” that areadjacent to the filter application pixel “y” in the vertical direction.Namely, the computing elements 93 a and 93 b calculate the imagecharacterizing quantities DIFF(y−1) and DIFF(y) defined by the equationF31 for the pixel values Y(y−1) to Y(y+1), respectively.

Each of comparators 94 a and 94 b receives one of the imagecharacterizing quantities DIFF(y−1) and DIFF(y) as a first input and thethreshold level TH1 of the vertical NR filter FH1 as a second input soas to produce one of the comparison results. Namely, each of thecomparators 94 a and 94 b compares one of the image characterizingquantities DIFF(y−1) and DIFF(y) with the threshold level TH1 of thevertical NR filter FH1 so as to deliver the comparison result to thedecision element 95 a. Each of comparators 94 c and 94 d receives one ofthe image characterizing quantities DIFF(y−1) and DIFF(y) as a firstinput and the threshold level TH2 of the vertical NR filter FH2 as asecond input so as to produce one of the comparison results. Namely,each of the comparators 94 c and 94 d compares one of the imagecharacterizing quantities DIFF(y−1) and DIFF(y) with the threshold levelTH2 of the vertical NR filter FH2 so as to deliver the comparison resultto the decision element 95 b. Each of comparators 94 e and 94 f receivesone of the image characterizing quantities DIFF(y−1) and DIFF(y) as afirst input and the threshold level-TH3 of the vertical NR filter FH3 asa second input so as to produce one of the comparison results. Namely,each of the comparators 94 e and 94 f compares one of the imagecharacterizing quantities DIFF(y−1) and DIFF(y) with the threshold levelTH3 of the vertical NR filter FH3 so as to deliver the comparison resultto the decision element 95 c.

The decision element 95 a receives outputs of the comparators 94 a and94 b and the threshold level TH_d1 for these outputs so as to calculatethe filter application condition F30. More specifically, if the pixels“y−1”, “y” and “y+1” are decided to be in a flat region (smooth region)for example, the value [1] is delivered so as to indicate that thevertical NR filter FH1 satisfies the filter application condition F30.The decision element 95 b receives outputs of the comparators 94 c and94 d and the threshold level TH_d2 for these outputs so as to calculatethe filter application condition F30. More specifically, if the pixels“y−1”, “y” and “y+1” are decided to be in a flat region (smooth region)for example, the value [1] is delivered so as to indicate that thevertical NR filter FH2 satisfies the filter application condition F30.The decision element 95 c receives outputs of the comparators 94 e and94 f and the threshold level TH_d3 for these outputs so as to calculatethe filter application condition F30. More specifically, if the pixels“y−1”, “y” and “y+1” are decided to be in a flat region (smooth region)for example, the value [1] is delivered so as to indicate that thevertical NR filter FH3 satisfies the filter application condition F30.

Here, the output of the decision element 95 a is delivered as theselection result 75 to the vertical NR process execution portion 58.Namely, if the output of the decision element 95 a indicates the value[1], it means that the vertical NR filter FH1 is used for the filterapplication pixel “y”. The output of the decision element 95 b isreceived by the AND gate 95 d. The AND gate 95 d receives negation ofthe output value of the decision element 95 a as a first input and theoutput value of the decision element 95 b as a second input so as todeliver the selection result 75. Namely, if the output of the decisionelement 95 a indicates the value [0] and the output of the decisionelement 95 b indicates the value [1], the AND gate 95 d delivers thevalue [1] so as to indicate that the vertical NR filter FH2 is used forthe filter application pixel “y”. The output of the decision element 95c is received by the AND gate 95 e. The AND gate 95 e receives negationof the output value of the decision element 95 a as a first input,negation of the output value of the decision element 95 b as a secondinput and the output value of the decision element 95 c as a third inputso as to deliver the selection result 75. Namely, if the outputs of thedecision elements 95 a and 95 b indicate the value [0] and the output ofthe decision element 95 c indicates the value [1], the AND gate 95 edelivers the value [1] so as to indicate that the vertical NR filter FH3is used for the filter application pixel “y”. In this way, the selectionresult 75 is delivered as a value that indicates whether each of thevertical NR filters FH1-FH3 is used or not.

FIG. 12 shows a hardware structure of the vertical NR process executionportion 58. The vertical NR process execution portion 58 receives thepixel values Y(y−1) to Y(y+1) at the same position in the horizontaldirection stored in three line memories as a first input and theselection result 75 as a second input so as to produce the NR processedsignal 67 (pixel value Y′(y)) that is obtained by the vertical NRprocess on the filter application pixel “y” of the pixel value Y(y)corresponding to the selection result 75. The vertical NR processexecution portion 58 includes adders 96 a and 96 b for performing addingoperation of signals, multipliers 97 a and 97 b for performingmultiplying operation of the filter factors, switches 98 a and 98 b fordesignating the filter factors, and a divider 100 for performingdividing operation of the horizontal NR processed signal 66 that ismultiplied by the filter factor.

The adder 96 a performs adding operation of pixel values having the samefilter factor that are multiplied even if any one of the vertical NRfilters FH1-FH3 is used. The adder 96 a adds the pixel value Y(y−1) andthe pixel value Y(y+1).

The multipliers 97 a and 97 b multiply respectively the outputs of theadder 96 a and the pixel value Y(y) of the filter application pixel “y”by the filter factor of the vertical NR filter indicated by theselection result 75 of the condition decision portion 57. The multiplier97 a receives the output of the adder 96 a as a first input and theoutput of the switch 98 a as a second input so as to multiply theseinputs. The multiplier 97 b receives the pixel value Y(y) of the filterapplication pixel “y” as a first input and the output of the switch 98 bas a second input so as to multiply these inputs.

The switches 98 a and 98 b operate in response to the selection result75 of the condition decision portion 57, and the operation of the switchdesignates the filter factor that is multiplied to the output of theadder 96 a and to the pixel value Y(y). The switches 98 a and 98 bdesignate the upper filter factor if the vertical NR filter FH1 is used,designate the middle filter factor if the vertical NR filter FH2 isused, and designate the lower filter factor if the vertical NR filterFH3 is used.

The adder 96 b adds the outputs of the multipliers 97 a and 97 b. Theadding result is divided by the sum of filter factors in the divider100. Here, the sum of filter factors is set to be the value [8] if anyone of the vertical NR filters FH1-FH3 is selected. Therefore, thedivider 100 can perform the dividing operation by shifting operation ofthree bits. The dividing result of the divider 100 is delivered as theNR processed signal 67.

(Filtering Method)

FIG. 13 shows a flowchart of a filtering method in the filter device 50,particularly a filtering method in the horizontal NR process portion 53and the vertical NR process portion 54.

This filtering method is a process that is performed for each pixel ofthe decoded image signal 65 received by the horizontal NR processportion 53 or each pixel received by the vertical NR process portion 54from line memories (not shown) (steps S20-S25). Hereinafter, a filteringmethod in the horizontal NR process portion 53 will be described. Notethat the process in the following description is the same in thevertical NR process portion 54.

The condition decision portion 55 sequentially decides whether or not avalue of the filter application condition F30 is [1] with regard to thehorizontal NR filters FH1-FH3 (step S21). The decision whether or notthe value of the filter application condition F30 is [1] is performed bythe process for each of the filter reference pixels (steps S21 a-S21 d).A value of DIFF(x+i) defined by the equation F31 is calculated withregard to “i” that is more than or equal to (−FHktap/2) and less than(FHktap/2), and DIFF(x+i) is compared with THk (“k” is the number of thecandidate filter that is an object in step S20) (step S21 b).Furthermore, using the comparison result, the filter applicationcondition defined by the equation F30 is calculated (step S21 d). If thereturn value [1] is produced, the filter FHk is used for the filterapplication pixel “x” (step S25), and the next process is started forthe next filter application pixel. On the other hand, if the returnvalue [0] is produced, a next candidate filter FHk+1 is prepared (stepsS22 and S23), and a similar process is performed (step S21). Inaddition, if the value of the filter application condition F30 is not[1] for each of the candidate filters, the horizontal NR filters FH1-FH3are not used, and the next process for the next filter application pixelis started.

The filtering method described above can be realized by a program usinga computer without being limited to the filter device 50.

(Effects of the Second Embodiment)

(1) One NR filter is selected for performing the NR process from theplurality of NR candidate filters in the filter device 50, so anappropriate NR effect can be obtained.

(2) It is possible to use a common method for calculating the imagecharacterizing quantity when selecting one NR filter from the pluralityof NR candidate filters. For example, it is possible to commonly use thecalculation results of the computing elements 81 a-81 f shown in FIG. 8and the computing elements 93 a and 93 b shown in FIG. 11 for selectingthe NR filters, so that hardware costs can be reduced and thesimplification of the process can be realized.

(3) When selecting one NR filter from the plurality of NR candidatefilters, pixels necessary for calculating the image characterizingquantity are the same as the pixels used by each of the plurality of NRfilters. For example, pixel values Y(x+3) to Y(x−3) are necessary forcalculating the image characterizing quantity by the condition decisionportion 55, and pixel values Y(x+3) to Y(x−3) are used by the horizontalNR process execution portion 56. Therefore, it is possible to structurethe delay elements 80 a-80 f and the delay elements 85 a-85 f as commondelay elements in FIGS. 8 and 9, so that hardware costs can be reducedand the simplification of the process can be realized.

In addition, pixel values Y(y−1) to Y(y+1) are necessary for thecondition decision portion 57 and the vertical NR process executionportion 58. Therefore, line memories for obtaining these pixel valuescan be structured as common line memories, so that hardware costs can bereduced and the simplification of the process can be realized.

(4) In the filter device 50, a stronger filter may be used as flatness(smoothness) of the image is larger. Therefore, an appropriate NR effectcan be obtained even if a contour of an object is included in the filterreference pixels.

(5) In the filter device 50, the sum of the filter factors of the NRfilters will be the same value. Therefore, it is possible to use acommon method for dividing the pixel values by the sum of the filterfactors. In addition, it is possible to set the sum of the filterfactors to [8] and to perform the dividing operation by shift operationof three bits. Therefore, the hardware structure can be simplified.

(6) In the filter device 50, as the filter intensity increases, thenumber of filter taps FHktap increases, and the threshold level THk ofthe filter application condition F30 decreases. Therefore, according tothe filtering method and the filtering program in filter device 50, thestructure of the software for using a filter having a higher NR effectwith higher priority can be simplified.

(Modifications of the Second Embodiment)

The present invention is not limited to the above embodiment, andvarious modifications are possible within the spirit of the presentinvention.

(1) Effects of the present invention are not limited to the case ofthree horizontal and vertical NR candidate filters. The number of the NRcandidate filters can be increased or decreased according to commonknowledge by changing the number of elements of hardware shown in FIGS.8, 9, 11 and 12.

(2) The NR candidate filter can be one having the number of filter tapsFHktap that does not increase and the threshold level THk of the filterapplication condition F30 does not decrease as the filter intensityincreases. Even in this case, it is possible to enhance the hardwarestructure of the filter device 50 shown as the second embodimentaccording to common knowledge.

(3) It is possible to use the vertical NR filter described above withreference to FIGS. 11 and 12 as the vertical NR filter portions 41-44shown in FIG. 2. In this case, it is possible to obtain the decisionresult 33 from the vertical NR process decision portion 20 and to add astructure which delivers the pixel value of the filter application pixelin accordance with the obtained result (filter through).

(4) The condition decision portion 57 may be one that performs thecondition decision by using the decoded image signal 65. In this case,the condition decision portion 57 can work in parallel with thehorizontal NR process before the horizontal NR process is completed, sothat the entire process speed of the filter device 50 can be improved.

THIRD EMBODIMENT

When selecting the NR filter and performing the filtering process, it ispossible to change the threshold level that is used in the filterapplication condition F30 or to use a filter for the block boundary inaccordance with whether or not the filter application pixel is locatedat the block boundary of the coded block. It is possible to realize thisfunction in the horizontal NR process portion 13 and the vertical NRprocess portion 14 shown in FIG. 1 and the horizontal NR process portion53 and the vertical NR process portion 54 shown in FIG. 5.

The horizontal NR process portion as a modified example of thehorizontal NR process portion 53 (see FIG. 5) will be described withreference to FIGS. 14-16. In the horizontal NR process portion as amodified example, selection of the NR filter and execution of the NRprocess are performed for the decoded image signal 65 that issequentially input by pixel position.

FIG. 14 shows numbers of filter taps and filter factors of threehorizontal NR filters FH1-FH3 that are used with being switched in thehorizontal NR process portion. The horizontal NR filter FH1 has thenumber of filter taps FH1tap=[5] and the filter factor [1, 2, 2, 2, 1].The horizontal NR filter FH1 is used only when the filter applicationpixel is a pixel that is adjacent to the block boundary. The horizontalNR filter FH2 has the number of filter taps FH2tap=[5] and the filterfactor [1, 1, 4, 1, 1]. The horizontal NR filter FH2 is used regardlessof whether or not the filter application pixel is a pixel that isadjacent to the block boundary. The horizontal NR filter FH3 has thenumber of filter taps FH3tap=[3] and the filter factor [2,4,2]. Thehorizontal NR filter FH3 is used only when the filter application pixelis not a pixel that is adjacent to the block boundary.

FIG. 15 shows a hardware structure of a condition decision portion 104as a modified example of the condition decision portion 55. Thecondition decision portion 104 receives the decoded image signal 65 as afirst input, a set threshold level 114 (TH1-TH3, THb) as a second input,and a block boundary flag 115 indicating that there is a block boundarybetween the pixel “x−2” and the pixel “x+2” (blk(x−2) to blk(x+1):blk(x) is a function that produces the value [1] if the block boundaryexists between the pixel “x” and the pixel “x+1”) as a third input,selects the horizontal NR filters FH1-FH3 that are used for the decodedimage signal 65, and produces the selection result 74. Here, the blockboundary flag 115 is information that is obtained by deciding whether ornot the pixel is on the block boundary in accordance with addressinformation of the pixel. The address information of a pixel is suppliedby supplying means such as a decoding device for decoding the decodedimage signal 65.

The condition decision portion 104 includes delay elements 105 a-105 dof one pixel delay, computing elements 106 a-106 d for calculating theimage characterizing quantity in accordance with pixel values ofneighboring pixels, comparators 107 a-107 j for realizing the comparisonfunction CMPk(x) defined by the equation F32, switches 108 a-108 f forswitching reference values to be given to the comparators 107 b, 107 c,107 e-107 h in accordance with the value of the block boundary flag 115,an OR gate 110, decision elements 111 a-111 c for realizing the filterapplication condition F30 and AND gates 111 d-111 f.

The delay elements 105 a-105 d are connected to the decoded image signal65 in series for one pixel delay, and produce pixel values Y(x+1) toY(x−2) of the past four pixels from the pixel value Y(x+2) received asthe decoded image signal 65.

Each of the computing elements 106 a-106 d calculates the imagecharacterizing quantity from the input and the output of each of thedelay elements 105 a-105 d. Namely, the computing elements 106 a-106 dcalculate the image characterizing quantities DIFF(x+1) to DIFF(x−2)defined by the equation F31 (see FIG. 6) for the pixel values Y(x+2) toY(x−2), respectively.

Each of comparators 107 a and 107 d receives one of the imagecharacterizing quantities DIFF(x+1) and DIFF(x−2) as a first input andthe threshold level TH1 of the horizontal NR filter FH1 as a secondinput so as to produce one of the comparison results. Namely, each ofthe comparators 107 a and 107 d compares one of the image characterizingquantities DIFF(x+1) and DIFF(x−2) with the threshold level TH1 of thehorizontal NR filter FH1 so as to deliver the comparison result to thedecision element 111 a. Each of comparators 107 b and 107 c receives oneof image characterizing quantities DIFF(x) and DIFF(x−1) as a firstinput and one of outputs of switches 108 a and 108 b as a second inputso as to produce one of the comparison results. Namely, the comparators107 b and 107 c compare the image characterizing quantities DIFF(x) andDIFF(x−1) with the outputs of switches 108 a and 108 b respectively soas to deliver the comparison result to the decision element 111 a.

Each of comparators 107 e-107 h receives one of the image characterizingquantities DIFF(x+1) to DIFF(x−2) as a first input and one of outputs ofswitches 108 c-108 f as a second input so as to produce one of thecomparison results. Namely, the comparators 107 e-107 h compare theimage characterizing quantities DIFF(x+1) to DIFF(x−2) with the outputsof the switches 108 c-108 f respectively so as to deliver the comparisonresult to the decision element 111 b.

Each of comparators 107 i and 107 j receives one of the imagecharacterizing quantities DIFF(x) and DIFF(x−1) as a first input and thethreshold level TH3 of the horizontal NR filter FH3 as a second input soas to produce one of the comparison results. Namely, the comparators 107i and 107 j compare the image characterizing quantities DIFF(x) andDIFF(x−1)) with the threshold level TH3 of the horizontal NR filter FH3so as to deliver the comparison result to the decision element 111 c.

Each of the switches 108 a and 108 b receives blk(x) or blk(x−1) of theblock boundary flag 115 as a first input, the threshold level TH1 of thehorizontal NR filter FH1 as a second input and a threshold level THb forthe block boundary as a third input so as to produce the threshold levelTH1 if blk(x) or blk(x−1) is the value [0] and to produce the thresholdlevel THb for the block boundary if blk(x) or blk(x−1) is the value [1].Each of the switches 108 c-108 f receives one of the (blk(x+1) toblk(x−2)) of the block boundary flag 115 as a first input, the thresholdlevel TH2 of the horizontal NR filter FH2 as a second input, and thethreshold level THb for the block boundary as a third input so as toproduce the threshold level TH2 if the first input is the value [0] andto produce the threshold level THb for the block boundary if the firstinput is value [1].

The OR gate 110 receives blk(x) of the block boundary flag 115 as afirst input and blk(x−1) of the same as a second input. Namely, the ORgate 110 produces the value [1] if the block boundary is adjacent to thefilter application pixel “x”. The output of the OR gate 110 is receivedby the AND gate 111 f. In addition, the negation of the output of the ORgate 110 is received by the AND gate 111 e.

The decision element 111 a receives the outputs of the comparators 107a-107 d and the threshold level TH_d1 for these output values. Morespecifically, if the pixels “x+2” to “x−2” are decided to be in a flatregion (smooth region) for example, the value [1] is produced so as toindicate that the horizontal NR filter FH1 satisfies the filterapplication condition. Here, the filter application condition issubstantially the same as the filter application condition F30 shown inFIG. 6, and the threshold level TH1 and the threshold level THb for theblock boundary are switched to be used as the reference value of thecomparison function CMPk(x) in the filter application condition F30.

The decision element 111 b receives the outputs of comparators 107 e-107h and the threshold level TH_d2 for these output values. Morespecifically, if the pixels “x+2” to “x−2” are decided to be in a flatregion (smooth region) for example, the value [1] is produced so as toindicate that the horizontal NR filter FH2 satisfies the filterapplication condition. Here, the filter application condition issubstantially the same as the filter application condition F30 shown inFIG. 6, and the threshold level TH2 and the threshold level THb for theblock boundary are switched to be used as the reference value of thecomparison function CMPk(x) in the filter application condition F30.

The decision element 111 c receives the outputs of comparators 107 i and107 j and the threshold level TH_d3 for these output values. Morespecifically, if the pixels “x+1” to “x−1” are decided to be in a flatregion (smooth region) for example, the value [1] is produced so as toindicate that the horizontal NR filter FH3 satisfies the filterapplication condition.

Here, the output of the decision element 111 a is received by the ANDgate 111 f. The AND gate 111 f receives the output value of the decisionelement 111 a as a first input and the output value of the OR gate 110as a second input so as to produce the selection result 74. Namely, ifthe output of the decision element 111 a indicates the value [1] and theoutput of the OR gate 110 indicates the value [1], the AND gate 111 fproduces the value [1] so as to indicate that the horizontal NR filterFH1 is used for the filter application pixel “x”.

The output of the decision element 111 b is received by the AND gate 11d. The AND gate 111 d receives negation of the output value of the ANDgate 111 f as a first input and the output value of the decision element111 b as a second input so as to produce the selection result 74.Namely, if the output of the AND gate 111 f indicates the value [0] andthe output of the decision element 111 b indicates the value [1], theAND gate 111 d produces the value [1] so as to indicate that thehorizontal NR filter FH2 is used for the filter application pixel “x”.

The output of the decision element 111 c is received by the AND gate 111e. AND gate 111 e receives negation of the output value of the AND gate111 f as a first input, negation of the output value of the decisionelement 111 b as a second input, the output value of the decisionelement 111 c as a third input and negation of the output value of theOR gate 110 as a fourth input so as to produce the selection result 74.Namely, if the outputs of the AND gate 111 f and the decision elements111 b indicate the value [0] and the output of the decision element 111c indicates the value [1] and the output of the OR gate 110 indicatesthe value [0], the AND gate 111 e produces the value [1] so as toindicate that the horizontal NR filter FH3 is used for the filterapplication pixel “x”. As described above, the selection result 74produces the value that indicates whether each of the horizontal NRfilters FH1-FH3 is used or not.

The selection result 74 is received by a horizontal NR process executionportion (not shown) as a modified example of the horizontal NR processexecution portion 56. The horizontal NR process execution portionreceives the decoded image signal 65 as a first input and the selectionresult 74 as a second input so as to produce the horizontal NR processedsignal 66 that is obtained by the horizontal NR process on the decodedimage signal 65 corresponding to the selection result 74. Here, astructure of the horizontal NR process execution portion issubstantially the same as the horizontal NR process execution portion 56shown in FIG. 9 except for the number of delay elements and the filterfactor, so description thereof will be omitted.

(Filtering Method)

FIG. 16 shows a flowchart of a filtering method in the horizontal NRprocess portion as a modified example of the horizontal NR processportion 53.

This filtering method is a process that is performed on each pixel inputin the horizontal NR process portion as the decoded image signal 65(steps S30-S35). The condition decision portion 104 sequentially decideswhether or not a value of the second filter application condition is [1]with regard to the horizontal NR filters FH1-FH3 (step S31).

Whether or not a value of the second filter application condition is [1]is decided in accordance with the process in steps S31 a-S31 l. First,it is decided whether the filter application pixel “x” is adjacent tothe block boundary or not (step S31 a).

If the filter application pixel “x” is not adjacent to the blockboundary, it is decided whether or not the horizontal NR filter FHk is ahorizontal NR filter that is used for a pixel that is not on the blockboundary (for example, the horizontal NR filter FH2 or the FH3 in FIG.14) (step S31 b). If the horizontal NR filter FHk is a horizontal NRfilter that is used only for a pixel on the block boundary (for example,the horizontal NR filter FH1 in FIG. 14), the return value [0] isproduced (step S31 c), and the process in steps S32-S34 is performed. Onthe other hand, if the horizontal NR filter FHk is a horizontal NRfilter that is used for a pixel that is not located on the blockboundary, the process in steps S31 f-S31 i for the filter referencepixel is performed.

If the filter application pixel “x” is a pixel on a block boundary (stepS31 a), it is decided whether or not the horizontal NR filter FHk is ahorizontal NR filter that is used for a pixel that is not on the blockboundary (step S31 d). If the horizontal NR filter FHk is a horizontalNR filter that is used only for a pixel that is not on the blockboundary, the return value [0] is produced (step S31 e), and the processin steps S32-step S34 is performed. On the other hand, if the horizontalNR filter FHk is a horizontal NR filter that is used for a pixel on theblock boundary, the process in steps S31 f-S31 l for the filterreference pixel is performed.

The condition decision portion 104 decides whether there is a blockboundary or not between the pixel “x+i” (−FHktap/2≦i<FHktap/2) and thepixel “x+i+1” (step S31 f).

If the decision result is negative, DIFF(x+i) defined by the equationF31 (see FIG. 6) is calculated, and DIFF(x+i) is compared with thethreshold level THk (“k” is the number of the candidate filter that isan object in step S30) (step S31 h).

If the decision result is positive, DIFF(x+i) defined by the equationF31 is calculated, and DIFF(x+i) is compared with the threshold levelTHb for the block boundary (step S31 i). Furthermore, using thecomparison result, the filter application condition defined by theequation F30 is calculated (step S31 k).

If the return value [0] is produced (step S31 l), the process in stepsS32-S34 is performed. In steps S32-S34, the next candidate filter FHk+1is prepared (steps S32 and S33), and the similar process is started(step S30).

If the return value [1] is produced (step S31 l), the filter FHk is usedfor the filter application pixel “x” (step S35), the process for thenext filter application pixel “x+1” is started.

The filtering method described above can be applied not only to thehorizontal NR process portion as a modified example of the horizontal NRprocess portion 53 but also to the horizontal NR process portion 13 andthe vertical NR process portion 14 shown in FIG. 1, and to the verticalNR process portion 54 shown in FIG. 5. In addition, it can be realizedby a program using a computer.

(Effects of the Third Embodiment)

(1) It is possible to use an NR filter for a block boundary havingstronger filter intensity for a pixel on the block boundary, forexample, in the filter device having the horizontal NR process portionor the vertical NR process portion as described in the third embodiment.Thus, it is possible to further reduce block noise.

(2) By setting the threshold level THb to a value larger than the normalthreshold levels TH1-TH3, it is possible to use the NR filters more forpixels on the block boundary. Thus, the block noise can be furtherreduced.

(3) In addition, by using an NR filter for a non-block boundary (forexample, the horizontal NR filter FH3 in FIG. 14), the NR process can beperformed using a more appropriate NR filter.

(4) In the condition decision portion 104 shown in FIG. 15, the imagecharacterizing quantity that is calculated for deciding usage of the NRfilter can be calculated more easily than a device for realizing theconventional filtering process shown in FIG. 25. Therefore, hardwarecosts can be reduced. In addition, with regard to a program thatperforms the same process by software, the process is easy and thus theprocess load can be reduced.

(Modifications of the Third Embodiment)

(1) In the horizontal NR filter shown in FIG. 14, the horizontal NRfilter FH1 is used only for a filter application pixel that is adjacentto a block boundary. Here, the horizontal NR filter for the blockboundary is not limited to when the filter application pixel is adjacentto the block boundary but may be used when it is close to the blockboundary with one pixel between them. In addition, it is possible toswitch filters in accordance with the number of pixels that existbetween the filter application pixel and the block boundary. By using afilter having higher filter intensity at a vicinity of the blockboundary, higher block noise can be reduced.

(2) Each of the candidate filters FH1-FH3 may use the threshold levelsTHb for the block boundary each of which has a different value.

FOURTH EMBODIMENT

As a fourth embodiment of the present invention, application examples ofa filter device, a filtering method and filtering program, and a systemusing the same will be described with reference to FIGS. 17-20.

FIG. 17 is a block diagram showing an overall structure of a contentproviding system ex100 that realizes a content delivering service. Anarea where a communication service is provided is divided into cells ofa desired size, and base stations ex107-ex110 that are fixed radiostations are provided in the cells.

This content providing system ex100 includes a computer ex111, apersonal digital assistant (PDA) ex112, a camera ex113, a cellular phoneex114, a cellular phone with camera ex115 and other equipment that areconnected to the Internet ex101 for example via an internet serviceprovider ex102, a telephone network ex104 and base stations ex107-ex110.

However, the content providing system ex100 can adopt any combinationfor connection without being limited to the combination shown in FIG.19. In addition, each of the devices can be connected directly to thetelephone network ex104 without the base stations ex107-ex110 that arefixed radio stations.

The camera ex113 is a device such as a digital video camera that canobtain a moving image. In addition, the cellular phone may be any typeof PDC (Personal Digital Communications) method, CDMA (Code DivisionMultiple Access) method, W-CDMA (Wideband-Code Division Multiple Access)method, or GSM (Global System for Mobile Communications) method, or acellular phone of PHS (Personal Handyphone System).

In addition, the streaming server ex103 is connected to the camera ex113via the base station ex109 and the telephone network ex104, so that livedelivery can be performed on the basis of coded data transmitted by auser of the camera ex113. The coding process of the obtained data may beperformed by the camera ex113 or by a server for transmitting data. Inaddition, the moving image data obtained by the camera ex116 may betransmitted to the streaming server ex103 via the computer ex111. Thecamera ex116 is a device that can take a still image like a digitalcamera and a moving image. In this case, coding of the moving image datamay be performed by the camera ex116 or by the computer ex111. Inaddition, the coding process may be performed by an LSI ex117 in thecomputer ex111 or the camera ex116. Note that it is possible toincorporate software for coding and decoding images into a storagemedium (a CD-ROM, a flexible disk, a hard disk or the like) that is arecording medium readable by the computer ex111. Furthermore, thecellular phone with camera ex115 may transmit the moving image data. Inthis case, the moving image data is coded by the LSI in the cellularphone ex115.

In this content providing system ex100, content (for example, a movingimage of a music concert) that the user is recording with the cameraex113 or the camera ex116 are coded and transmitted to the streamingserver ex103, while the streaming server ex103 delivers a stream of thecontent data to a client who made a request. The client may be thecomputer ex111, the PDA ex112, the camera ex113, the cellular phoneex114 or the like that can decode the coded data. Thus, in the contentproviding system ex100, the client can receive and reproduce the codeddata. The system can realize personal broadcasting when the clientreceives, decodes and reproduces the stream in real time. In addition,reproduction of content may be performed by using the filter device, thefiltering method and the filtering program of the embodiment describedabove. For example, the computer ex111, the PDA ex112, the camera ex113and the cellular phone ex114 may be equipped with the filter device andthe filtering program described in the above embodiment.

The cellular phone will be exemplified for the following description.

FIG. 18 shows the cellular phone ex115 that utilizes a media datadisplay device of the above embodiment. The cellular phone ex115includes an antenna ex201 for transmitting and receiving radio waveswith the base station ex110, a camera portion ex203 such as a CCD camerathat can take a still image, a display portion ex202 such as a liquidcrystal display for displaying images obtained by the camera portionex203 or images received by the antenna ex201 after the image data aredecoded, a main body portion including a group of operating keys ex204,a sound output portion ex208 such as a speaker for producing sounds, asound input portion ex205 such as a microphone for receiving sounds, arecording medium ex207 for storing coded data or decoded data such asdata of taken moving images or still images, data of received e-mails,moving images or still images, and a slot portion ex206 that enables therecording medium ex207 to be attached to the cellular phone ex115. Therecording medium ex207 such as an SD card includes a plastic casehousing a flash memory element that is one type of EEPROM (ElectricallyErasable and Programmable Read Only Memory) nonvolatile memory that iselectronically rewritable and erasable.

Furthermore, the cellular phone ex115 will be described with referenceto FIG. 19. The cellular phone ex115 includes a main controller portionex311 for controlling each portion of the main body portion having thedisplay portion ex202 and the operating keys ex204, a power sourcecircuit portion ex310, an operational input controller portion ex304, animage coding portion ex312, a camera interface portion ex303, an LCD(Liquid Crystal Display) controller portion ex302, an image decodingportion ex309, a multiplex separation portion ex308, a recordreproduction portion ex307, a modem circuit portion ex306 and a soundprocessing portion ex305, which are connected to each other via asynchronizing bus ex313.

When the user turns on a clear and power key, the power source circuitportion ex310 supplies power from a battery pack to each portion so thatthe digital cellular phone with camera ex115 is activated.

The cellular phone ex115 converts a sound signal collected by the soundinput portion ex205 during a sound communication mode into digital sounddata by the sound processing portion ex305 under control of the maincontroller portion ex311 that includes a CPU, a ROM and a RAM. Thedigital sound data are processed by the modem circuit portion ex306 as aspectrum spreading process and are processed by the transmission andreception circuit portion ex301 as a digital to analog conversionprocess and a frequency conversion process. After that, the data aretransmitted via the antenna ex201. In addition, the cellular phone ex115amplifies a signal that is received by the antenna ex201 during thesound communication mode and performs the frequency conversion processand an analog to digital conversion process on the data, which isprocessed by the modem circuit portion ex306 as a spectrum reversespreading process and is converted into a analog sound signal by thesound processing portion ex305. After that, the analog sound signal isdelivered by the sound output portion ex208.

Furthermore, when transmitting electronic mail during a datacommunication mode, text data of the electronic mail are entered byusing the operating keys ex204 of the main body portion and are given tothe main controller portion ex311 via the operational input controllerportion ex304. The main controller portion ex311 performs the spectrumspreading process on the text data by the modem circuit portion ex306and performs the digital to analog conversion process and the frequencyconversion process by the transmission and reception circuit portionex301. After that, the data are transmitted to the base station ex110via the antenna ex201.

When transmitting image data during the data communication mode, theimage data obtained by the camera portion ex203 are supplied to theimage coding portion ex312 via the camera interface portion ex303. Inaddition, if the image data are not transmitted, it is possible todisplay the image data obtained by the camera portion ex203 directly bythe display portion ex202 via the camera interface portion ex303 and anLCD controller portion ex302.

The image coding portion ex312 converts the image data supplied from thecamera portion ex203 into the coded image data by compressing and codingthe data, and the coded image data are supplied to the multiplexseparation portion ex308. In addition, the cellular phone ex115 collectssounds by the sound input portion ex205 while the camera portion ex203is taking the image, and the digital sound data is supplied from thesound processing portion ex305 to the multiplex separation portionex308.

The multiplex separation portion ex308 performs multiplexing of thecoded image data supplied from the image coding portion ex312 and thesound data supplied from the sound processing portion ex305 by apredetermined method. Multiplexed data obtained as a result areprocessed by the modem circuit portion ex306 as a spectrum spreadingprocess and are processed by the transmission and reception circuitportion ex301 as a digital to analog conversion process and a frequencyconversion process. After that, the data are transmitted via the antennaex201.

When receiving moving image file data that are linked to a web pageduring the data communication mode, a signal received from the basestation ex110 via the antenna ex201 is processed by the modem circuitportion ex306 as a spectrum reverse spreading process. Multiplexed dataobtained as a result are supplied to the multiplex separation portionex308.

In addition, in order to decode multiplexed data received via theantenna ex201, the multiplex separation portion ex308 separates a codedbit stream of image data in the multiplexed data from a coded bit streamof sound data. Then, the multiplex separation portion ex308 supplies thecoded image data to the image decoding portion ex309 via thesynchronizing bus ex313 and supplies the sound data to the soundprocessing portion ex305.

Next, the image decoding portion ex309 generates reproduction movingimage data by decoding the coded bit stream of the image data andsupplies the data to the display portion ex202 via the LCD controllerportion ex302. Thus, the moving image data included in a moving imagefile that is linked to a home page can be displayed.

In addition, the image decoding portion ex309 may function as a filterdevice described in the above embodiment.

In this case, the sound processing portion ex305 converts the sound datainto an analog sound signal, which is supplied to the sound outputportion ex208. Thus, sound data included in the moving image file thatis linked to a home page can be reproduced.

Note that the present invention is not limited to the example of thesystem described above. Digital broadcasting by satellite or terrestrialsignals has been a recent topic of discussion. As shown in FIG. 20, thefilter device, the filtering method and the filtering program of theabove embodiment can be incorporated into the digital broadcastingsystem, too. More specifically, in a broadcast station ex409, a codedbit stream of image information is sent to a communication or abroadcasting satellite ex410 via a radio wave. The broadcastingsatellite ex410 that received the coded bit stream of image informationsends radio waves for broadcasting. These radio waves are received by anantenna ex406 of a house equipped with a satellite broadcastingreception facility, and a device such as a television set (a receiver)ex401 or a set top box (STB) ex407 decodes the coded bit stream andreproduces the same. Here, the television set (the receiver) ex401 orthe set top box (STB) ex407 may be equipped with the filter device ofthe above embodiment. In addition, it may use the filtering method ofthe above embodiment. Furthermore, it may have the filtering program ofthe above embodiment. In addition, a reproduction device ex403 forreading and decoding a coded bit stream that is recorded on a storagemedium ex402 such as a CD or a DVD that is a recording medium may beequipped with the filter device, the filtering method and the filteringprogram of the above embodiment. In this case, the reproduced imagesignal is displayed on a monitor ex404. In addition, it is possible tomount a filter device of the above embodiment in a set top box ex407that is connected to a cable ex405 for a cable television or the antennaex406 for a satellite or surface wave broadcasting, so that the imagecan be reproduced on a monitor ex408 of the television set. In thiscase, it is possible to incorporate the filter device not into the settop box but into the television set. In addition, it is possible that acar ex412 equipped with an antenna ex411 receives a signal from thebroadcasting satellite ex410 or the base station ex107 and reproducesthe moving image on a display of a navigation system ex413 in the carex412.

Furthermore, it is possible to code the image signal, which is recordedin a recording medium. As a specific example, there is a recorder ex420such as a DVD recorder for recording image signals on a DVD disk ex421or a disk recorder for recording image signals on a hard disk.Furthermore, it is possible to record on an SD card ex422. In addition,it is possible to reproduce image signals recorded on a DVD disk ex421or a SD card ex422 via a filter device of the above embodiment, so as todisplay on the monitor ex408.

Note that in the structure of the navigation system ex413 shown in FIG.19, the camera portion ex203, the camera interface portion ex303 and theimage coding portion ex312 can be omitted. This can be also applied tothe computer ex111 and the television set (the receiver) ex401.

In addition, the terminal device such as the cellular phone ex114 mayinclude three types of assemblies. A first type is a transmission andreception terminal having both the coder and the decoder, a second typeis a transmission terminal having only a coder and a third type is areception terminal having only a decoder.

Thus, the filter device, the filtering method and the filtering programof the above embodiment can be used for any device and system describedabove, so that effects described above can be obtained.

Note that each functional block in block diagrams (shown in FIGS. 1, 2,5 and 19) and structures of hardware (FIGS. 8, 9, 11, 12 and 15) aretypically realized as an LSI that is an integrated circuit. These may beintegrated in a plurality of chips or a single chip. For example,functional blocks except the memory may be integrated in a single chip.

The LSI may be referred to as an IC, a system LSI, a super LSI or anultra LSI in accordance with the degree of integration.

In addition, a method for integrating circuits is not limited to an LSIbut it may be realized by an application specific integrated circuit ora versatile processing unit. It is possible to use an FPGA (FieldProgrammable Gate Array) that is programmable after the LSI is producedor a silicon figurable processor that can restructure connection orsetting of circuit cells in the LSI.

Furthermore, if another technique for integrating circuits rather thanhe LSI appears with the progress of semiconductor technology, then thattechnique may be utilized for integrating the functional blocks.Biotechnology has the potential for such technology.

The filter device according to the present invention is useful as afilter device in which the reduction of hardware costs is desired,particularly as a filter device that utilizes noise reduction (NR)filters. In addition, the filtering method and the filtering programaccording to the present invention is useful as a filtering method and afiltering program that utilize NR filters in which the reduction ofhardware costs or the reduction of the software process load is desired.

1. A filter device for performing a filter process on a plurality ofproceed signals obtained by processing an input image signal that hasbeen input, the filter device comprising: an NR process decision unitoperable to determine a decision result value in accordance with aresolution conversion filter selection signal; an NR process unit havinga plurality of NR filter units and operable to output at least one ofthe plurality of processed signals through a direct path, output aremainder of the plurality of processed signals through the plurality ofNR filter units, the plurality of NR filter units being operable todetermine whether or not to perform a filtering process in accordancewith the decision result value, and output the at least one of theplurality of processed signals output the direct path and the remainderof the plural it of processed signals output through the plurality of NRfilter units as NR processed signals; and a resolution conversion unitoperable to convert resolution of the NR processed signals to generate adisplay image signal in accordance with the resolution conversion filterselection signal, and output the display image signal, wherein said NRprocess decision unit is operable to switch ON or OFF of the NR filterunits in accordance with an operation of said resolution conversionunit, the operation of the resolution conversion being determined basedon the resolution conversion filter selection signal; wherein if theresolution conversion filter selection signal indicates to convertresolution by using the plurality of processed signals corresponding topixels less than the pixels to which the plurality of processed signalscorrespond, said NR process decision unit determines the decision resultvalue to be a value indicating NR filter ON; wherein if the resolutionconversion filter selection signal indicates to convert resolution byusing the plurality of processed signals corresponding to pixels equalto the pixels to which the plurality of processed signals corresponds,said NR process decision unit determines the decision result value to bea value indicating NR filter OFF; and wherein if the resolutionconversion filter selection signal indicates not to convert resolution,said NR process decision unit determines the decision result value to bea value indicating NR filter ON.
 2. A filter device for performing afilter process on a plurality of processed signals obtained byprocessing an input image signal that has been input, the filter devicecomprising: an NR process decision unit operable to determine a decisionresult value in accordance with a resolution conversion filter selectionsignal: an NR process unit having a plurality of NR filter units andoperable to output at least one of the plurality of processed signalsthrough a direct path, output a remainder of the plurality of processedsignals through the plurality of NR filter units, the plurality of NRfilter units being operable to determine whether or not to perform afiltering process in accordance with the decision result value, andoutput the at least one of the plurality of processed signals outputthrough the direct path and the remainder of the plurality of processedsignals output through the plurality of NR filter units as NR processedsignals; and a resolution conversion unit operable to convert resolutionof the NR processed signals to generate a display image signal inaccordance with the resolution conversion filter selection signal, andoutput the display image signal, wherein said NR process decision unitis operable to switch ON or OFF of the NR filter units in accordance witan operation of said resolution conversion unit, the operation of theresolution conversion being determined based on the resolutionconversion filter selection signal; wherein said NR process unitincludes a horizontal NR process unit and a vertical NR process unit,the horizontal NR process unit is operable to receive an input imagesignal, perform a horizontal NR process on the input image signal andoutput a plurality of horizontal NR processed signals, said NR processdecision unit includes a vertical NR process decision unit operable todetermine a decision result value in accordance with a resolutionconversion filter selection signal; wherein the vertical NR process unithas a plurality of vertical NR filter units and operable to output atleast one of the plurality of horizontal NR processed signals through adirect path, output a remainder of the plurality of horizontal NRprocessed signals through the plurality of vertical NR filter units, theplurality of vertical NR units being operable to determine whether ornot to perform a vertical filtering process in accordance with thedecision result value, and output the at least one of the plurality ofhorizontal NR processed signals output through the direct path and theremainder of the plurality of horizontal NR processed signals outputthrough the plurality of vertical NR filter units as NR processedsignals; wherein a vertical resolution conversion unit convertsresolution of the NR processed signals to generate a display imagesignal in accordance with the resolution conversion filter selectionsignal, and outputs the display image signal; wherein if verticalresolution conversion is not performed, the resolution conversion filterselection signal is set to indicate operation of no resolutionconversion; wherein if vertical resolution conversion is a conversionwith a simple conversion ratio such as an integer or an inverse numberof an integer, the resolution conversion filter selection signal is setto indicate operation of a middle M line input and one line output; andwherein if vertical resolution conversion is not a conversion with asimple conversion ratio such as an integer or an inverse number of aninteger, the resolution conversion filter selection signal is set toindicate operation of line input and one line output, where the numberof line input when vertical resolution conversion is a conversion with asimple conversion ratio is smaller than the number of line input whenvertical resolution conversion is not a conversion with a simpleconversion ratio.